Advanced techniques in scientific computing: application to electromagnetics

Mención Internacional en el título de doctorDurante los últimos años, los componentes de radiofrecuencia que forman parte de un sistema de comunicaciones necesitan simulaciones cada vez más exigentes desde el punto de vista de recursos computacionales. Para ello, se han desarrollado diferentes técnicas con el método de los elementos finitos (FEM) como la conocida como adaptatividad hp, que consiste en estimar el error en el problema electromagnético para generar mallas de elementos adecuadas al problema que obtienen una aproximación de forma más efectiva que las mallas estándar; o métodos de descomposición de dominios (DDM), basado en la división del problema original en problemas más pequeños que se pueden resolver en paralelo. El principal problema de las técnicas de adaptatividad es que ofrecen buenas prestaciones para problemas bidimensionales, mientras que en tres dimensiones el tiempo de generación de las mallas adaptadas es prohibitivo. Por otra parte, DDM se ha utilizado satisfactoriamente para la simulación de problemas eléctricamente muy grandes y de gran complejidad, convirtiéndose en uno de los temas más actuales en la comunidad de electromagnetismo computacional. El principal objetivo de este trabajo es estudiar la viabilidad de algoritmos escalables (en términos de paralelización) combinando DDM no conformes y adaptatividad automática en tres dimensiones. Esto permitir ía la ejecución de algoritmos de adaptatividad independiente en cada subdominio de DDM. En este trabajo se presenta y discute un prototipo que combina técnicas de adaptatividad y DDM, que aún no se han tratado en detalle en la comunidad científica. Para ello, se implementan tres bloques fundamentales: i) funciones de base para los elementos finitos que permitan órdenes variables dentro de la misma malla; ii) DDM no conforme y sin solapamiento; y iii) algoritmos de adaptatividad en tres dimensiones. Estos tres bloques se han implementado satisfactoriamente en un código FEM mediante un método sistemático basado en el método de las soluciones manufacturadas (MMS). Además, se ha llevado a cabo una paralelización a tres niveles: a nivel de algoritmo, con DDM; a nivel de proceso, con MPI (Message Passing Interface); y a nivel de hebra, con OpenMP; todo en un código modular que facilita el mantenimiento y la introducción de nuevas características. Con respecto al primer bloque fundamental, se ha desarrollado una familia de funciones base con un enfoque sistemático que permite la expansión correcta del espacio de funciones. Por otra parte, se han introducido funciones de base jerárquicas de otros autores (con los que el grupo al que pertenece el autor de la tesis ha colaborado estrechamente en los últimos años) para facilitar la introducción de diferentes órdenes de aproximación en el mismo mallado. En lo relativo a DDM, se ha realizado un estudio cuantitativo del error generado por las disconformidades en la interfaz entre subdominios, incluidas las discontinuidades generadas por un algoritmo de adaptatividad. Este estudio es fundamental para el correcto funcionamiento de la adaptatividad, y no ha sido evaluado con detalle en la comunidad científica. Además, se ha desarrollado un algoritmo de adaptatividad con prismas triangulares, haciendo especial énfasis en las peculiaridades debidas a la elección de este elemento. Finalmente, estos tres bloques básicos se han utilizado para desarrollar, y discutir, un prototipo que une las técnicas de adaptatividad y DDM.In the last years, more and more accurate and demanding simulations of radiofrequency components in a system of communications are requested by the community. To address this need, some techniques have been introduced in finite element methods (FEM), such as hp adaptivity (which estimates the error in the problem and generates tailored meshes to achieve more accuracy with less unknowns than in the case of uniformly refined meshes) or domain decomposition methods (DDM, consisting of dividing the whole problem into more manageable subdomains which can be solved in parallel). The performance of the adaptivity techniques is good up to two dimensions, whereas for three dimensions the generation time of the adapted meshes may be prohibitive. On the other hand, large scale simulations have been reported with DDM becoming a hot topic in the computational electromagnetics community. The main objective of this dissertation is to study the viability of scalable (in terms of parallel performance) algorithms combining nonconformal DDM and automatic adaptivity in three dimensions. Specifically, the adaptivity algorithms might be run in each subdomain independently. This combination has not been detailed in the literature and a proof of concept is discussed in this work. Thus, three building blocks must be introduced: i) basis functions for the finite elements which support non-uniform approximation orders p; ii) non-conformal and non-overlapping DDM; and iii) adaptivity algorithms in 3D. In this work, these three building blocks have been successfully introduced in a FEM code with a systematic procedure based on the method of manufactured solutions (MMS). Moreover, a three-level parallelization (at the algorithm level, with DDM; at the process level, with message passing interface (MPI), and at the thread level, with OpenMP) has been developed using the paradigm of modular programming which eases the software maintenance and the introduction of new features. Regarding first building block, a family of basis functions which follows a sound mathematical approach to expand the correct space of functions is developed and particularized for triangular prisms. Also, to ease the introduction of different approximation orders in the same mesh, hierarchical basis functions from other authors are used as a black box. With respect to DDM, a thorough study of the error introduced by the non-conformal interfaces between subdomains is required for the adaptivity algorithm. Thus, a quantitative analysis is detailed including non-conformalities generated by independent refinements in neighbor subdomains. This error has not been assessed with detail in the literature and it is a key factor for the adaptivity algorithm to perform properly. An adaptivity algorithm with triangular prisms is also developed and special considerations for the implementation are explained. Finally, on top of these three building blocks, the proof of concept of adaptivity with DDM is discussed.Programa Oficial de Doctorado en Multimedia y ComunicacionesPresidente: Daniel Segovia Vargas.- Secretario: David Pardo Zubiaur.- Vocal: Romanus Dyczij-Edlinge

Herramienta de simulación remota en un cluster de computacion científica

Debido a la aparición de problemas cada vez más complejos y pesados computacionalmente hablando, resulta casi indispensable recurrir al uso de clusters de altas prestaciones, o HPCC, para reducir sensiblemente el tiempo de ejecución o ser capaces de abordar el inmenso tamaño de estos. Sin embargo, para la mayoría de usuarios el acceso a estos HPCC es tedioso e, incluso, complicado. Para superar esta barrera de entrada, en este Proyecto Fin de Carrera se describe una herramienta segura, sencilla y fácil de usar para proporcionar el acceso a los servicios de uno de estos clusters. De esta forma, se implementa un protocolo específico de comunicaciones con el cual se ofrecen funcionalidades como la ejecución remota de problemas en un HPCC o la transferencia automática de los ficheros de entrada y salida del trabajo ejecutado. Asimismo, se ha desarrollado una interfaz gráfica basada en Java para facilitar el control de estas funcionalidades por el usuario desde un equipo de sobremesa y, además, se ha implementado una aplicación en Android que mantiene los mismos servicios que la interfaz de sobremesa en un dispositivo móvil, dotando así de movilidad a la aplicación desarrollada. _________________________________________________________________________________________________________________________As a result of the existence of problems more and more complex and computationally expensive, it is a must to use of high performance computational clusters, or HPCC, in order so as to appreciably reduce the time of execution and be capable of solve such problems. However, for a novice user the access to one of this resources can be di fficult and even unpleasant. This Final Project is developed to solve this barrier to entry. A simple, secure and user-friendly tool to provide the access to the services of one of these clusters is described. Thus, a speci c communication protocol is implemented, which provides features as the remote execution of problems in a HPCC or the automatic file transfer of the input and output les of the executed job. Likewise, a GUI (Graphical User Interface) based on Java has been developed to simplify these features in a desktop computer or laptop. And also, an Android application has been implemented in order to increase the mobility keeping all features from the original GUI.Ingeniería de Telecomunicació

Construction of higher-order curl-conforming finite elements and its assembly

Different choices are available when constructing vector finite element bases in real coordinates. In this communication, two different designs of higher-order curl-conforming basis functions are introduced and explained, showing the particularities of its assembly. Tetrahedra and hexahedra are used as element shapes to assess the effect of triangular and quadrilateral faces on the two considered constructions of basis functions. A comparison of their robustness in terms of the condition number of the finite element matrices for a number of distortions is includedMinisterio de Ciencia y Tecnología, Grant/Award Numbers: TEC2013- 47753-C3, TEC2016-80386-P; Ministerio de Educación, Cultura y Deporte, Grant/Award Number: FPU14/0374

Adaptive Semi-Structured Mesh Refinement Techniques for the Finite Element Method

The adaptive mesh techniques applied to the Finite Element Method have continuously been an active research line. However, these techniques are usually applied to tetrahedra. Here, we use the triangular prismatic element as the discretization shape for a Finite Element Method code with adaptivity. The adaptive process consists of three steps: error estimation, marking, and refinement. We adapt techniques already applied for other shapes to the triangular prisms, showing the differences here in detail. We use five different marking strategies, comparing the results obtained with different parameters. We adapt these strategies to a conformation process necessary to avoid hanging nodes in the resulting mesh. We have also applied two special rules to ensure the quality of the refined mesh. We show the effect of these rules with the Method of Manufactured Solutions and numerical results to validate the implementation introduced.This work has been financially supported by TEC2016-80386-

Test-Driven Development of a Substructuring Technique for the Analysis of Electromagnetic Finite Periodic Structures

In this paper, we follow the Test-Driven Development (TDD) paradigm in the development of an in-house code to allow for the finite element analysis of finite periodic type electromagnetic structures (e.g., antenna arrays, metamaterials, and several relevant electromagnetic problems). We use unit and integration tests, system tests (using the Method of Manufactured Solutions—MMS), and application tests (smoke, performance, and validation tests) to increase the reliability of the code and to shorten its development cycle. We apply substructuring techniques based on the definition of a unit cell to benefit from the repeatability of the problem and speed up the computations. Specifically, we propose an approach to model the problem using only one type of Schur complement which has advantages concerning other substructuring techniques.This work has been financially supported by TEC2016-80386-P and PID2019-109984RB-C41

Second-Order Nedelec Curl-Conforming Prismatic Element for Computational Electromagnetics

A systematic approach to obtaining mixed-order curl-conforming basis functions for a triangular prism is presented; focus is made on the second-order case. Space of functions for the prism is given. Basis functions are obtained as the dual basis with respect to suitably discretized Nedelec degrees of freedom functionals acting on elements of the space. Thus, the linear independence of the basis functions is assured while the belonging of the basis to the a priori given space of functions is guaranteed. Different strategies for the finite element assembly of the basis are discussed. Numerical results showing the verification procedure of the correctness of the implemented basis functions are given. Numerical results about sensibility of the condition number of the basis obtained concerning the quality of the elements of the mesh are also shown. Comparison with other representative sets of basis functions for prisms is included.This work was supported by "DiDaCTIC: Desarrollo de un sistema de comunicaciones inalambrico en rango THz integrado de alta tasa de datos"; TEC2013-47753-C3, CAM S2013/ICE-3004 "DIFRAGEOS" projects and "Ayudas para contratos predoctorales de Formación del Profesorado Universitario FPU

Meshing strategies for 3d geo-electromagnetic modeling in the presence of metallic infrastructure

In 3D geo-electromagnetic modeling, an adequate discretisation of the modeling domain is crucial to obtain accurate forward responses and reliable inversion results while reducing the computational cost. This paper investigates the mesh design for subsurface models, including steel-cased wells, which is relevant for many exploration settings but still remains a numerically challenging task. Applying a goal-oriented mesh refinement technique and subsequent calculations with the high-order edge finite element method, simulations of 3D controlled-source electromagnetic models in the presence of metallic infrastructure are performed. Two test models are considered, each needing a distinct version of approximation methods to incorporate the conductive steel casings of the included wells. The influence of mesh quality, goal-oriented meshing, and high-order approximations on problem sizes, computational cost, and accuracy of electromagnetic responses is investigated. The main insights of our work are: (a) the applied numerical schemes can mitigate the computational burden of geo-electromagnetic modeling in the presence of steel artifacts; (b) investigating the processes driving the meshing of models with embedded metallic infrastructures can lead to adequate strategies to deal with the inversion of such electromagnetic data sets. Based on the modeling results and analyses conducted, general recommendations for modeling strategies are proposed when performing simulations for challenging steel infrastructure scenarios.Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. The work of O.C-R. has received funding from the Ministerio de Educación y Ciencia (Spain) under Project TED2021-131882B-C42.The code development of P.R. has been financed by the Smart Exploration project. Smart Exploration has received funding from the European Union’s Horizon 2020 Framework Programme under grant agreement N∘ 775971. The computations were enabled by resources provided by the Swedish National Infrastructure for Computing (SNIC) at UPPMAX partially funded by the Swedish Research Council through grant agreement N∘ SNIC 2021/22-883.Peer ReviewedPostprint (published version

GPU Acceleration of a Non-Standard Finite Element Mesh Truncation Technique for Electromagnetics

The emergence of General Purpose Graphics Processing Units (GPGPUs) provides new opportunities to accelerate applications involving a large number of regular computations. However, properly leveraging the computational resources of graphical processors is a very challenging task. In this paper, we use this kind of device to parallelize FE-IIEE (Finite Element-Iterative Integral Equation Evaluation), a non-standard finite element mesh truncation technique introduced by two of the authors. This application is computationally very demanding due to the amount, size and complexity of the data involved in the procedure. Besides, an efficient implementation becomes even more difficult if the parallelization has to maintain the complex workflow of the original code. The proposed implementation using CUDA applies different optimization techniques to improve performance. These include leveraging the fastest memories of the GPU and increasing the granularity of the computations to reduce the impact of memory access. We have applied our parallel algorithm to two real radiation and scattering problems demonstrating speedups higher than 140 on a state-of-the-art GPU.This work was supported in part by the Spanish Government under Grant TEC2016-80386-P, Grant TIN2017-82972-R, and Grant ESP2015-68245-C4-1-P, and in part by the Valencian Regional Government under Grant PROMETEO/2019/109

Strategies to parallelize a finite element mesh truncation technique on multi-core and many-core architectures

Achieving maximum parallel performance on multi-core CPUs and many-core GPUs is a challenging task depending on multiple factors. These include, for example, the number and granularity of the computations or the use of the memories of the devices. In this paper, we assess those factors by evaluating and comparing different parallelizations of the same problem on a multiprocessor containing a CPU with 40 cores and four P100 GPUs with Pascal architecture. We use, as study case, the convolutional operation behind a non-standard finite element mesh truncation technique in the context of open region electromagnetic wave propagation problems. A total of six parallel algorithms implemented using OpenMP and CUDA have been used to carry out the comparison by leveraging the same levels of parallelism on both types of platforms. Three of the algorithms are presented for the first time in this paper, including a multi-GPU method, and two others are improved versions of algorithms previously developed by some of the authors. This paper presents a thorough experimental evaluation of the parallel algorithms on a radar cross-sectional prediction problem. Results show that performance obtained on the GPU clearly overcomes those obtained in the CPU, much more so if we use multiple GPUs to distribute both data and computations. Accelerations close to 30 have been obtained on the CPU, while with the multi-GPU version accelerations larger than 250 have been achieved.Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work has been supported by the Spanish Government PID2020-113656RB-C21, PID2019-106455GB-C21 and by the Valencian Regional Government through PROMETEO/2019/109, as well as the Regional Government of Madrid throughout the project MIMACUHSPACE-CM-UC3M

3D Magnetotelluric Modeling Using High-Order Tetrahedral Nédélec Elements on Massively Parallel Computing Platforms

We present a routine for 3D magnetotelluric (MT) modeling based upon high-order edge finite element method (HEFEM), tailored and unstructured tetrahedral meshes, and high-performance computing (HPC). This implementation extends the PETGEM modeller capabilities, initially developed for active-source electromagnetic methods in frequency-domain. We assess the accuracy, robustness, and performance of the code using a set of reference models developed by the MT community in well-known reported workshops. The scale and geological properties of these 3D MT setups are challenging, making them ideal for addressing a rigorous validation. Our numerical assessment proves that this new algorithm can produce the expected solutions for arbitrarily 3D MT models. Also, our extensive experimental results reveal four main insights: (1) high-order discretizations in conjunction with tailored meshes can offer excellent accuracy; (2) a rigorous mesh design based on the skin-depth principle can be beneficial for the solution of the 3D MT problem in terms of numerical accuracy and run-time; (3) high-order polynomial basis functions achieve better speed-up and parallel efficiency ratios than low-order polynomial basis functions on cutting-edge HPC platforms; (4) a triple helix approach based on HEFEM, tailored meshes, and HPC can be extremely competitive for the solution of realistic and complex 3D MT models and geophysical electromagnetics in general.This project has been 65% cofinanced by the European Regional Development Fund (ERDF) through the Interreg V-A Spain–France– Andorra program (POCTEFA2014-2020). POCTEFA aims to reinforce the economic and social integration of the French–Spanish–Andorran border. Its support is focused on developing economic, social and environmental cross-border activities through joint strategies favoring sustainable territorial development. BSC authors received funding from the European Union’s Horizon 2020 programme, grant agreement N◦828947 and N◦777778, and from the Mexican Department of Energy, CONACYT-SENER Hidrocarburos grant agreement N◦B-S-69926